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This study explored EFL students and faculty's 
perceptions of and attitudes towards the use of 
online assessment and practice. A descriptive 
method was used, employing quantitative data 
collection and analyses from a sample of 400 
students of different age categories and edu¬ 
cational levels and another sample of 25 teach¬ 
ers in the English department. Results showed 
that students' attitudes towards e-testing 
are generated by the test-takers' perceptions 
of self-efficacy, enjoyment, usefulness, behav¬ 
ioural intentions to use web-based assessment, 
system satisfaction and system challenges 
in the order of the responses of the students. 
Furthermore, attitudes towards e-testing 
are generated by the students' salient beliefs 
about the consequences of continued use and 
the evaluation of online testing technology. 
Web-based assessment and practice provided 
students with immediate feedback and auto¬ 
mated scores that help students to have more 
control over their work and their effort. The 
findings from this study imply that creating 
web-based assessment technology awareness, 
motivation, and changing faculty and learners' 
behaviours and attitudes is required for the 
success of e-testing adoption, implementation 
and diffusion in the future. 

Keywords: EFL, online testing, LMS, teacher 
attitudes, learner attitudes 

Introduction 

Educational assessment is a significantly 
integral phase of learning that is concomi¬ 
tant with the different phases of learning 
initially in placement and diagnostic tests, 
medially as a formative procedure and 
finally as in summative measurement of 41 
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learning outcomes. Furthermore, assessment can be a cyclical process in which the learn¬ 
ers could be assessed periodically in order to check on their progress using home assign¬ 
ments and observation of learning procedures. 

Learning outcomes, or achievement, assessment can be incorporated in a quiz, a home¬ 
work assignment, a set of questions, a test, a project, a portfolio, or a presentation. With 
developments in learning delivery platforms, and the massive use of e-learning, assessment 
is also becoming widely done online. By definition, online assessment is method of using 
networked computers to deliver and analyse both achievement and performance tests. 

Online testing, also known as “Computer Assisted Assessment (CAA), is that sector of 
e-learning aimed at assessing learners' knowledge through e-learning means” (Costagliola 
& Fuccella, 2009, p. 63), being independent of time and place, and more significantly, cost- 
effective and efficient. 

Generally, online testing, often including e-tests and homework assignments, can play 
a crucial role in e-learning as it can reinforce learning. Online assessment procedures can 
potentially promote students' self-discipline, independence and responsibility characteris¬ 
tics (McPherson, 2005). The aim of an assignment is to teach students how to study inde¬ 
pendently, plan for learning and test-taking efficiently, and get organized in group assign¬ 
ments help promote thinking individually and collaboratively (Sgouros & Martin, 2005). 

The growing use of online testing as a global trend and the fast-growing introduction 
of online assessment in Saudi education systems has engendered an interest in conducting 
systematic research into the effectiveness of online testing in terms of efficiency and useful¬ 
ness (AlNajdi, 2014). This study seeks to answer urgent call for needed research on online 
assessment, especially in the context of Saudi Arabia. A thorough search in databases for 
research done to evaluate online educational assessment in Saudi Arabia revealed nearly 
no single study on the topic. This paucity in research on e-testing in Saudi Arabia speaks 
loudly to the need for systematic empirical research in the area of online assessment in 
English language teaching to recognize the perceptions and attitudes of students and fac¬ 
ulty in the most leading e-learning university in Saudi Arabia. 

Research question 

The main research question in the present study is: 

)Jc What are the EFL students and faculty's perceptions of and attitudes towards the use of 
online assessment and practice in a Saudi university? 

This question bore out the following sub-questions: 

1. How do EFL students and faculty perceive of the self-efficacy of the online assessment 
system? 

2. How do EFL students and faculty perceive of the usefulness of the online assessment 
system? 

3. How do EFL students and faculty perceive of the system challenges of the online assess¬ 
ment system? 

4. How do EFL students and faculty perceive of their enjoyment of the online assessment 
system? 

5. How do EFL students and faculty perceive of their behavioural intentions to use online 
assessment procedures? 
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6. How do EFL students and faculty perceive of their satisfaction with the online assess¬ 
ment system? 

Purpose of the study 

Upon recent suggestions of the use of online testing methods in enhancing language teach¬ 
ing and learning and attitudes thereto, this study seeks to explore the students and fac¬ 
ulty perceptions of online assessment and practice as regards language learning attitudes. 
Explicating unique capabilities of the innovative online assessment practices on making 
significant contributions to traditional evaluation practice would be eventually conducive 
to recommendations for massive future use and appropriate development of online assess¬ 
ment and tutorial system like Blackboard. 

Literature review 

Research on college-level online learning, though spare and sparse, as analysed in the 
synthesis study by Patrick & Powell (2009), revealed "no significant difference" in student 
performance in online courses versus traditional face-to-face courses; and in particular 
programmes, researchers also found that students learning online are performing “equally 
well or better” than their peers in traditional instructional media (Patrick & Powell, 2009, p. 
8).Consequently, e-learning evaluation studies further suggested other factors that interfere 
with the adoption of technology for classroom practices' acceptance and adoption of new 
systems or services; of such are the following: (1) user satisfaction (e.g., Arbaugh, 2000; 
Burns, Clift & Duncan, 1990; Hsu, Yen, Chiu, & Chang, 2006; Liao, Chen, & Yen, 2007), 

(2) learners’ and teachers’ motivation and attitudes (e.g., Bhattacherjee & Sanford, 2006; 
Mekheimer, 2012; Ushida, 2005), (3) perceived usefulness (e.g., Roca, Chiu, & Martinez, 
2006; Liao, et al., 2007; Gefen, 2003; Hsu and Lu, 2004), (4) perceived ease of use (e.g., Roca, 
et al., 2006), and (5) quality (e.g., Chiu, et al., 2005; McKinney et al., 2002; Roca et al., 2006). 
However, the relationship between the initial technology acceptance and post-adoption of 
a service of e-learning has seldom been investigated, especially in the Arab world (Fageeh, 
2011). 

Currently, several educational researchers (Chang, Hsu & Fang, 2010; Hristova, 2008; 
Sokolova & Totkov, 2005) predicted that e-testing will eventually replace conventional 
testing procedures, thus becoming the main assessment activity at universities. Thus, the 
need for efficient e-testing environments and methods, as well as good authorware and 
instructors for their support arises urgently. 

Bennet (2001) further describes online assessment as an "interactive, broadband, net¬ 
worked, and standard-based” evaluation instrument. The case being thus, online assessment 
can afford a large number of randomised items with immediate scoring and feedback, a 
rapid return of a large number of tests and survey results (Bennet, 2001; Linn, 2002) as 
well as provide an automatic diagnosis system for performance evaluation (Nguyen & 
Kulm, 2005). A few decades ago, with the emergence of Java, JavaScript and HTML, online 
assessment has become an interactive medium for delivering both formative and summa- 
tive assessment, given the myriad of benefits associated with online testing. According to 
Hovland (2005), online testing is more affordable, efficient, and can measure language 
learning skills unmeasurable on conventional tests under less stress. In addition, online 
data storage facilitates testing management and security of data through data encryption. 43 
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As such, the convenience, rapidity, and efficiency of online testing will gradually lead to the 
partial disappearance of traditional testing methods, being widely replaced by online tests. 

Further, and above all, online testing can provide students with more opportunities 
"for practice, self-testing, self-regulation, and self-evaluation while teachers receive feed¬ 
back from students, save time in reading and grading, and eventually have closer inter¬ 
actions with students." (Nguyen, Hsieh, & Allen, 2006, p. 252). Some research indicated 
that by resorting to online assessment, teachers can integrate different content areas with 
language instruction and enhance learners' attitudes and motivation towards learning 
(Fleischman, 2001; Morgan & O'Reilly, 2001; Nguyen, Hsieh, Jane & Allen, 2006). 

EFL students' attitudes towards learning vary based on characteristics of classroom 
factors such as instruction and assessment and their delivery methods and tools (Hart & 
Walker, 1993). In addition, Steele & Arth (1998) indicated that the flexibility of accepting 
students’ ways of solving problems as is the case in online testing can increase students' par¬ 
ticipation, reduce anxiety, and induce positive attitudes for enhanced learning. Web-based 
applications harnessed for both instruction and assessment can create different learning 
and evaluation contexts as well as produce flexible approaches to learning, teaching and 
assessment (Allen, 2001; Liang & Creasy, 2004). Flexible teaching and learning are learner- 
based, less time-and-place-dependent than traditional forms of teaching, and are designed 
to increase the learner’s responsibility for their own learning. Flexible learning and teach¬ 
ing also caters for variation in individual differences and can induce self-directed learning 
practice. In addition, flexible assessment gets rid of face-to-face contact that causes test 
anxiety, always makes use of technology, saves time and money, and provides immediate 
feedback which is essential for effective learning. 

According to Middleton & Spanias (1999, cited in Nguyen, et al., 2006, p. 254), "assess¬ 
ment provided on the web-based medium allows students to have more control over their 
practice and to receive immediate reinforcement that can help build their intrinsic motiva¬ 
tion and improve their confidence". Across other disciplines such as science and mathemat¬ 
ics inasmuch as the case in content-integrated language teaching, researchers concurred 
that online assessment applications could back up and improve students’ levels of confi¬ 
dence, motivation, engagement and interaction (Galbraith & Haines, 1998; Alderson, 2000; 
2006). Additionally, web-based assessment has the potential to meet the authentic assess¬ 
ment standards (ones that are amenable to assessment in an observable and measurable 
fashion) defined by Kulm (1994) which serve learning objectives, namely: 1) improvement 
of instruction and learning; 2) providing feedback for the students to aid the in observing 
to inappropriate strategies, thinking or habits; and 3) improvement of attitudes towards 
learning (Kulm, 1994, p. 4). In that way, online assessment can help reinforce learning 
objectives by providing immediate feedback, reinforcing appropriate learning strategies, 
enhancing thinking habits, and eventually inducing enhanced attitudes, and as such, can 
help meet the demands of authentic assessment. 

Previous research on the integration of online assessment technology into language 
learning and teaching has shown promising and positive effects in relation to attitude 
improvement and enhancement of motivation as well as improved performance (Johnston 
& Barker, 2002; Shi, 2012; Brantmeier, 2006; Chalhoub-Deville, 2001; Chapelle, 2001; Choi, 
Kim & Boo, 2003; Dikii, 2006; Goodwin-Jones, 2001; Hemard & Cushion, 2003). 
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Limitations of online assessment technologies 

Despite its flexibility and malleability, web-based assessment has several limitations. For 
instance, students must have reliable access to networked computers without time restric¬ 
tions. Some technical hitches, such as sluggish internet connection, network jams, or sud¬ 
den disconnections may render these assessment tools ineffective. 

Web-based assessment does not provide partial credits and cannot sufficiently grade 
short answers and essay guestions, either. Research also indicated that the level of computer 
familiarity or technological literacy also affects achievement on online tests (Brantmeier, 
2006; Choi, et al., 2003; Dikii, 2006; Elder, et al., 2007; Hegelheimer & Chapelle, 2000; Kern, 
1995; Liou, 2000; Roever, 2001; Thurmond, et al., 2002). 

In addition, Nguyen, et al. (2006, p. 255) aptly observed that “to meet the current 
demands of the technological era, web-based practice has continued to be developed, and 
has perpetually improved its unigue features and characteristics" - an observation that 
resonates with prior research findings and recommendations (e.g., Alderson & McIntyre, 
2006; Coniam, 2003; Gorsuch, 2004) through introducing help options and multimedia 
incorporated into the online interactive test (Hegelheimer & Chapelle, 2000), investigat¬ 
ing more effective methods of responding to student papers (Krucli, 2004), developing and 
trying bilingual computerized test of vocabulary size (Laufer& Goldstein, 2004), applying 
dynamic assessment, which focuses on process rather than on the product, to CMC lan¬ 
guage development (Oskoz, 2005),using input-environment outcome assessment model 
to control for student characteristics in order to decrease bias (Thurmond, et al., 2002; 
Volchok, et al., 2006), and understanding intercultural communication, mediated by cul¬ 
tural artefacts (i.e., internet communication tools), for creating compelling, problematic, 
and surprising conditions for additional language learning (Thorne, 2003; Thorne & Payne, 
2005; Weber & Abel, 2003). 

Reviews of several years of development in computer-based testing are now accruing, 
touching on transfers from paper-based to computer-based tests, computer adaptive testing, 
the use of corpus data to increase authenticity, and natural language processing develop¬ 
ments in machine scoring of written language (Jamieson, 2005). These reviews have also 
discussed the pros and cons of assessment of learner strategies in CALL (Liou, 2000). 

Notwithstanding the defects of and cons against online testing methods, the web-based 
technology is now emerging as one of the best media of user interface and instructional 
delivery and assessment. Web-based assessment, eventually, lends itself to constitute an 
integral part of the curriculum and learning process (Allen, 2001; Heift, & Schulze, 2003; 
Hemard, & Cushion, 2003). The benefits of web-based assessment inspire researchers to 
take endeavours to evaluate this inevitably new alternative to the traditional paper-based 
tests that are going obsolescent nowadays. 

Hypotheses 

The present study sought to verify the following hypotheses: 

1. There is as statistical difference between students’ responses and rankings of the factors 
influencing their perceptions of their attitudes towards e-testing across age categories 
of students. 

2. There is a statistical difference between faculty's attitudes towards online testing across 
teaching/testing experience using new technology. 


45 




The JALT CALL Journal 2015: Regular Papers 


Methodology 

Method 

This study applied a descriptive method employing quantitative data collection and analy¬ 
ses methods through the administration of two instruments: Students’ Perceptions and 
Attitudes towards Online Testing Survey and Faculty's Perceptions and Attitudes towards 
Online Testing Survey. 

Sample 

The Students' Perceptions and Attitudes towards Online Testing Survey was distributed to a 
sample of 400 students of different age categories and educational levels randomly drawn 
from the student population of the English programme of the Faculty of Languages & 
Translation, King Khalid University. The Faculty's Perceptions and Attitudes towards Online 
Testing Survey was submitted online to a random sample of 25 teachers in the English 
department of the same institution. 

Reliability and validity of the instruments 

To determine the internal validity of the Students’ Perceptions and Attitudes towards Online 
Testing Survey, a Cronbach alpha was calculated on a sample of EFL learner population (N 
=400), using SPSS (alpha = 0.971). 

Further, to determine the internal validity of the Faculty's Perceptions and Attitudes 
towards Online Testing Survey, a Spearman-Brown Coefficient was calculated by distributing 
the questionnaire to a sample of teachers involved in e-teaching (N =25). The Spearman- 
Brown Coefficient was calculated using SPSS (alpha = 0.963C 

Results 

Informants of the study (N = 400) fall into an array of age categories, with most of them 
being in the age range from 20 years of age to 23 years of age enrolled in advanced levels 
of the English programme. 

Students' attitudes towards online testing 

As the tables below show, it appears that most respondents do not feel the test jitters upon 
using e-learning environments or labs for taking their e-tests. 

Perceived self-efficacy. Self-efficacy refers to test takers' judgment of their own ability to 
perform specific tasks (Lee, 2006). 
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Table 1: Perceived self-efficacy 


Perceived self-efficacy 

Mean 

SD 

Minimum 

Maximum 

1 feel confident taking an online test. 

3.80 

1.158 

1 

5 

1 feel confident using the Blackboard system for taking 
different types of online tests, including essay tests and 
objective tests. 

3.56 

1.289 

1 

5 

1 do not feel the test jitters associated with using 
e-learning environments or labs for taking my e-tests. 

3.86 

1.152 

1 

5 


Table 1 data show that students, in general, do not feel test anxiety when they take their 
tests using e-testing labs, then on the second rank, they feel confident taking online tests, 
while the least rank goes for feeling confident using Blackboard tools for taking different 
web-based assessment tasks. Indeed, no matter how advanced or powerful a technology 
is, its effective implementation depends upon users having positive attitudes towards it as 
well as having higher levels of self-efficacy in using computers for teaching and learning 
(Wu, Tennyson, & Hsia, 2010). 

Perceived enjoyment. Perceived enjoyment refers to the students' perceptions of the ease of 
use of the e-testing system as well as to their perceptions of the content guality. Perceived 
ease of use (PEOU) of an e-learning and/or e-testing system is defined as the degree to 
which an individual believes that using a particular technology for learning or for testing 
will be fun and free of effort (Lee, 2006; Davis, 1989). On the other hand, perceived content 
guality refers to the assumption that information guality is significant in determining 
users’ level of satisfaction with the system, which, in turn, leads to system utilisation for 
teaching and assessment. There are two dimensions of content guality: 'content richness’, 
and ‘update regularity’. The first of these, “content richness", positively affects learners’ or 
test-takers’ levels of satisfaction with the course/e-test (Burns et al., 1990; Arbaugh, 2000). 
The richness of an e-course or an e-test content can be improved by providing additional 
educational/psychometric software on a CD-ROM. In other words, the learners' perceptions 
of the content guality and ease of use related to the online testing system shape the learn¬ 
ers’ attitudes towards its use and behavioural intention to make use of an e-testing system. 

As can be seen in Table 2 below, the students showed that they enjoyed using the 
Blackboard system for test-taking most, and in general, they enjoy using computers as a 
test-taking assistive tool in second rank, and third, they enjoy using the Grade Centre for 
knowing about their results. 

This aspect of attitudes towards e-testing - perceived enjoyment - postulates that usage 
behaviours of individuals towards online testing are shaped by the experiences with the 
technology (Agarwal & Karahanna, 2000; Davis, 1989; Davis, Bagozzi, & Warshaw, 1989; 
Lederer et al., 1998). On the other, perceived ease of use is an individual's assessment that 
technology interaction will be relatively free of cognitive burden, i.e., ease of use reflects 
the facility with which the individual is able to interact with a particular software artefact. 
Overall, perceived enjoyment and perceived self-efficacy indicated positive attitudes towards 
e-testing as well as heightened intrinsic motivation for using a computer-mediated system 
(CMS) for regular web-based assessment. This finding echoes prior research that related 


47 




The JALT CALL Journal 2015: Regular Papers 


perceived ease of use and enjoyment to enhanced e-teaching experience (e.g. ; Roca, et al., 
2006; Venkatesh, 2000). 


Table 2: Perceived enjoyment 


Perceived enjoyment 

M 

SD 

Min 

Max 

1 enjoy using computers as a testing-taking assistive tool. 

4.07 

.986 

1 

5 

1 enjoy using e-learning environment for testing purposes. 

4.18 

1.080 

1 

5 

1 enjoy using the Blackboard Grade Centre for testing-taking. 

3.84 

1.097 

1 

5 


Perceived usefulness. The perceived usefulness of a teaching/testing online system is 
defined as the extent to which individuals believe that using the new technology will 
enhance their task performance (Lee, 2006; Davis, 1989). 

Results shown in Table 3 below demonstrate that students believe that using online 
testing technology is useful for taking language assessments the most while they believe 
the least that using the Blackboard environment for e-learning purposes can be helpful 
for language assessment. They strongly believe that using e-learning environments, over¬ 
all, is helpful for online test taking. Prior research indicated that perceived usefulness and 
perceived enjoyment have direct positive effects on the intention to use an LMS efficiently, 
as they have direct positive effects on the intention and motivation to use e-learning and 
e-testing technologies - findings commensurate with those of prior research (Lee, 2010; 
Roca and Gagne, 2008). 


Table 3: Perceived usefulness 


Perceived usefulness 

M 

SD 

Min 

Max 

1 believe using e-learning environments is helpful for language 

assessment. 

3.75 

1.190 

1 

5 

1 believe using e-learning environments is helpful for e-testing. 

4.03 

1.144 

1 

5 

1 believe online assessment is useful for taking language tests. 

4.10 

1.042 

1 

5 


Behavioural intention to use e-testing. According to Lee (2006, pp. 519), within an e-learn¬ 
ing context, “the adoption of ELS is a positive function of the intention (Bl) to accept the 
system." Given the significance of affective factors in predicting and improving e-learning, 
e-teaching and e-testing (See for example, Liaw, et al., 2007), new issues and considerations 
related to post-adoption of e-learning tools and continuance intention on the part of lan¬ 
guage teachers, particularly the role of attitudes and motivation, have recently given rise 
to nascent research stamina (Abdullah, Abidin, Luan, Majid & Atan, 2006; Lin, 2011; Lonn, 
et al., 2011; Sugar, Crawley, & Fine, 2004). 

Results in Table 4 indicate that students showed behavioural intentions to use the 
Grade Centre to train on major and minor testing, and then in the second rank, they 
showed intentions to use online assessment facilities to access the procedures and out¬ 
comes of their regular evaluation. Third, they showed the least behavioural intentions to 
use Blackboard’s Grade Centre to check their grades on major and minor assessments and 
48 assignments. Prior research indicates that perceptions of the usefulness and ease of use of 
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e-learning technology influences students’ attitudes towards behavioural intentions and 
actual use of CMSs (Fageeh, 2011) as are teachers’ being influenced by extrinsic motivation 
and confirmation of pre-acceptance expectations (Sorebo, et ah, 2009). 


Table 4: Behavioural intention to use e-testing 


Behavioural intention to use e-testing 

M 

SD 

Min 

Max 

1 intend to use the Grade Centre to check grades on major and 
minor assessments. 

3.92 

1.115 

1 

5 

1 intend to use the facilities of the Grade Centre. 

4.17 

.965 

1 

5 

1 intend to use the Grade Centre to train on major and minor 
testing. 

4.24 

.913 

1 

5 


Perceived system satisfaction. Positive attitudes towards e-testing technology has previ¬ 
ously been associated with perceived system satisfaction (Hsu, Yen, Chiu, & Chang, 2006; 
Liao, Chen, & Yen, 2007). According to Lee (2006), several rigorous studies suggested that 
“information guality is important in determining users’ level of satisfaction with the system, 
which in turn, leads to system utilisation" (p. 521). 

Results from the present research showed that informants of the study indicated that 
they are satisfied most with using Blackboard online testing tools, while they are satisfied 
least with using the facilities of the Grade Centre to take valid and reliable language assess¬ 
ments. They showed also satisfaction with using e-learning environments and/or testing 
labs for language web-based assessment in the second rank. This result is consistent with 
some previous studies (Burns et ah, 1990; Arbaugh, 2000). 


Table 5: Perceived system satisfaction 


Perceived system satisfaction 

M 

SD 

Min 

Max 

1 am satisfied with using e-learning environments and/or testing 
labs for language test-taking. 

3.96 

1.319 

1 

5 

1 am satisfied with using the facilities of the Grade Centre to 
take valid and reliable language assessments. 

3.93 

1.074 

1 

5 

1 am satisfied with using the online assessments via Blackboard. 

4.06 

1.084 

1 

5 


Perceived system challenges. E-learning and e-testing systems are fraught with a plethora 
of challenges and demands that reguire a more rigorous engineering approach, testing and 
re-testing with real users, in order to meet these demands. Of these challenges are testing 
schedules, and the testers' as well as testees' timetables, internet connectivity hitches, inter¬ 
ruptions and assistance online with test-taking. Results from the present research showed 
that informants of the study indicated that test scheduling difficulties come in the first rank 
as perceived problems by the students, then come discipline problems during testing ses¬ 
sions, such as noise, confusion or cheating, then come connectivity problems, and finally 
problems seeking help with test administrators during testing sessions. 
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Table 6: Perceived system challenges 


Perceived system challenges 

M 

SD 

Min 

Max 

1 find difficulty scheduling my time to match the tester's 
timetables for e-testing. 

3.81 

1.162 

1 

5 

My colleagues are uncontrollable in terms of discipline 

3.78 

1.092 

1 

5 

There are Internet connectivity problems/technical hitches that 
may interrupt the test-taking process. 

3.88 

1.035 

1 

5 

1 rarely receive help from the test administrators in the e-testing 
labs. 

3.85 

1.081 

1 

5 


Summary of student survey findings. Students' attitudes towards e-testing are generated 
by the test-takers’ perceptions of self-efficacy, enjoyment, usefulness, behavioural inten¬ 
tions to use web-based assessment, system satisfaction and system challenges in the order 
of the responses of the students. Attitudes towards e-testing are generated by the students' 
salient beliefs about the consequences of continued use and the evaluation of these conse¬ 
quences (Ajzen, 1988). Such beliefs regarding self-efficacy, enjoyment, usefulness, system 
satisfaction and the challenges and outcomes of use provide strong internal incentives for 
the adoption of e-testing (Davis, et al., 1989). These factors be generally divided into three 
major types of measurements: affective, cognitive and behavioural. 

Table 7: Order of the factors impacting students' attitudes towards e-testing 


Factors/Survey sections 

Mean 

Std. Deviation 

Perceived self-efficacy 

3.74 

1.20 

Perceived enjoyment 

4.03 

1.05 

Perceived usefulness 

3.96 

1.13 

Behavioural intention to use e-testing 

4.11 

1.00 

Perceived system satisfaction 

3.98 

1.16 

Perceived system challenges 

3.83 

1.09 


Perceived system challenges 
16% 


Perceived system satisfaction 
16% 


Behavioural intention to use e-testing 
16% 



Perceived self-efficacy 
16% 


Perceived enjoyment 
16% 


Perceived usefulness 
16% 
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Figure 1: Students' attitudes towards online testing: order of influential factors 
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Effect of age. The researcher ran a one-way AN OVA to analyze the differences between 
students’ means of their responses to the factors affecting their attitudes towards e-testing 
designated by the six sections of the survey and their associated procedures (such as "varia¬ 
tion'' among and between groups) to see if there were any differences between informants 
on the variable of age. Results are presented in Table 8 below: 


Table 8: ANOVA for the factors affecting students' attitudes towards e-testing by age 


Factors impacting 

Source of 

Sum of 


Mean 



students' attitudes 

variance 

squares 

df 

square 

F 

Sig. 

Perceived enjoyment 

Between Groups 

740.571 

2 

370.286 

60.391 

.000 


Within Groups 

2434.189 

397 

6.131 




Total 

3174.760 

399 




Perceived usefulness 

Between Groups 

999.663 

2 

499.832 

74.970 

.000 


Within Groups 

2646.834 

397 

6.667 




Total 

3646.497 

399 




Behavioural intention 

Between Groups 

698.637 

2 

349.318 

67.557 

.000 

to use e-testing 

Within Groups 

2052.761 

397 

5.171 




Total 

2751.398 

399 




Perceived system 

Between Groups 

978.637 

2 

489.319 

80.563 

.000 

challenges 

Within Groups 

2411.260 

397 

6.074 




Total 

3389.898 

399 




Perceived system 

Between Groups 

1951.774 

2 

975.887 

95.670 

.000 

challenges 

Within Groups 

4049.624 

397 

10.201 




Total 

6001.398 

399 




Attitudes towards 

Between Groups 

39305.906 

2 

19652.953 

102.069 

.000 

online evaluation 

Within Groups 

76440.934 

397 

192.546 




Total 

115746.840 

399 





As shown in Table 10 above, the AN OVA tests revealed that the students' attitudes towards 
online testing according to their age indicate that there is as statistical difference between 
students’ responses and rankings of the factors influencing their perceptions of their atti¬ 
tudes towards e-testing across age categories if the informants. 

Faculty's perceptions and attitudes towards online testing survey 

The following tables show the frequencies and percentages of teachers’ responses by the 
variables of age and experience; the perceptions and attitudes’ survey for faculty was admin¬ 
istered to 25 faculty members of varied ages and experience. Analysis of demographic data 
of faculty respondents indicate that 40% of the faculty sample were between 30 and 40 
years of age, 36% were more than 36 years old, and 24% were less than 30 years old. Further 
analysis of demographic data of faculty indicate that 48% of the sample were of experi¬ 
ence above 10 years, 28% were of experience less than 5 years, and 24% were of experience 
from 5 to 10 years. 
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Results of faculty analysis of their perceptions and attitudes towards online testing. The 

following tables show the mean scores, standard deviation and maximum versus minimum 
scores for each item on each section of the faculty’s survey; it appears that the higher the 
score, the more freguently respondents do not feel the test jitters associated with using 
e-testing environments or labs for administering or giving their e-tests to their students. 


Table 11: Perceived self-efficacy 


Perceived self-efficacy 

M 

SD 

Min 

Max 

1 feel confident administering online assessment. 

3.80 

.707 

2 

5 

1 feel confident using Blackboard for testing students. 

3.40 

.764 

2 

5 

1 feel confident using e-learning environments for e-testing. 

3.80 

.764 

2 

5 


Perceived self-efficacy. This aspect of informants' attitudes indicate that perceived self-effi¬ 
cacy refers to test administrators "/test makers' judgment of their own ability to administer 
specific testing tasks during testing sessions. Data in Table 11 show that test-administrators 
perceive self-confidence giving web-based tests is ranked of top significance, followed by 
confidence using the Blackboard for e-testing and confidence using e-testing labs for con¬ 
ducting e-testing sessions. 

Perceived enjoyment. This aspect of informants' attitudes indicate that perceived enjoy¬ 
ment refers teachers' perceptions of the ease of use of the e-testing system as well as to 
their perceptions of test administration in e-testing labs. Data in Table 12 show that test- 
administrators enjoy using computers as a test-making assistive tool, as well as using the 
e-testing environment for testing purposes, and finally using Blackboard Grade Centre for 
test-making and results announcements. 


Table 12: Perceived enjoyment 


Perceived enjoyment 

Mean 

SD 

Minimum 

Maximum 

1 enjoy using computers as an assistive testing tool. 

4.04 

.841 

2 

5 

1 enjoy using e-learning for testing purposes. 

3.72 

.843 

2 

S 

1 enjoy using the Grade Centre for test-making. 

3.88 

.833 

3 

5 


Perceived usefulness. Lee (2006) defines perceived usefulness of using web-based assess¬ 
ment as the beliefs of users, in this case, teachers’ assumptions that using the new tech¬ 
nology will enhance their task performance. Results in Table 13 show that most teachers 
believe that web-based assessment environments are helpful for language evaluation; in 
general, these environments are also useful for testing for different purposes in the lan¬ 
guage classroom, and are finally, they are useful for conducting language assessment that 
follows e-learning as in web-based assignment and home-work tasks. 
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Table 13: Perceived usefulness 


Perceived usefulness 

Mean 

SD 

Minimum 

Maximum 

1 believe using e-learning environments is helpful for 
language assessment. 

4.00 

.957 

1 

5 

1 believe using e-learning environments is helpful for 
testing. 

3.88 

.881 

2 

5 

1 believe using online instruction is useful for conducting 
language assessments. 

4.36 

.810 

3 

5 


Perceived behavioural intention to use e-testing. Lee (2006) defines behavioural intention 
to use e-testing environments as the adoption of new technology being a positive func¬ 
tion of the intention of teachers to accept the system for web-based assessment. Results 
in Table 14 show that most teachers have intentions to use Blackboard's Grade Centre to 
assist them in conducting their online tests and in doing different types of evaluation, for¬ 
mative and summative. 


Table 14: Behavioural intention to use e-testing 


Behavioural intention to use e-testing 

Mean 

SD 

Minimum 

Maximum 

1 intend to use Blackboard's Grade Centre to assist my 
testing. 

3.48 

.918 

2 

5 

1 intend to use online assessment to assist my evaluation. 

4.08 

.812 

3 

5 

1 intend to use the Grade Centre to assist my testing. 

4.04 

.841 

3 

5 


Perceived system satisfaction. This determinant of teachers' attitudes towards online test¬ 
ing refers to users’ level of satisfaction with the system, which in turn, leads to system 
utilisation. Results in Table 15 show that most teachers are satisfied with using web-based 
assessment technology in labs, including the facilities of the Grade Centre, which help in 
validating tests and keeping them reliable for use. 


Table 15: system satisfaction 


Perceived system satisfaction 

Mean 

SD 

Minimum 

Maximum 

1 am satisfied with using e-learning environments and/or 
testing labs for language testing. 

3.12 

1.166 

1 

5 

1 am satisfied with using the facilities of the Grade Centre 
to conduct valid and reliable language assessments. 

4.00 

.913 

2 

5 

1 am satisfied with using online assessments via 

Blackboard. 

3.80 

.866 

2 

5 


Perceived system challenges. This factor impacting teachers' attitudes towards online test¬ 
ing refers to the difficulties using online testing technology. Results in Table 16 show 
that most teachers have a great difficulty with booking e-testing labs due to paucity in 53 













The JALT CALL Journal 2015: Regular Papers 


web-based facilities; as well, there is an expansion in using web-based assessments as an 
increasing testing medium. Next come problems with keeping order and discipline during 
e-testing sessions, then connectivity and technical problems and finally seeking assistance 
from lab technicians. 


Table 16: Perceived system challenges 


Perceived system challenges 

M 

SD 

Min 

Max 

1 find difficulty booking an e-testing lab due to heavy use of the 
labs along the semester. 

4.04 

.841 

3 

5 

Students are uncontrollable in terms of discipline in the e-testing 
labs. 

3.80 

.764 

3 

5 

There are Internet connectivity problems and other technical 
hitches that may interrupt a test and spoil it. 

3.80 

.764 

3 

5 

1 rarely get technical assistance from the e-testing lab technician. 

3.20 

.866 

2 

5 


Summary of faculty survey findings. Teachers' attitudes towards e-testing are generated by 
the test-makers' perceptions of self-efficacy, enjoyment, usefulness, behavioural intentions 
to use web-based assessment, system satisfaction and system challenges in the order of the 
responses of teachers. This order is akin to the order of responses of informant students 
which shows some kind of similitude in the perceptions of teachers and students as to the 
main factors impacting attitudes towards online testing. Attitudes towards e-testing are 
generated by the teachers’ significant beliefs about continued evaluation using web-based 
assessment technology. Such beliefs regarding self-efficacy, enjoyment, usefulness, system 
satisfaction and the challenges and outcomes of use provide strong internal incentives for 
the adoption of e-testing on the part of teachers as it is on the part of students (Davis, et 
ah, 1989). 

Table 17: Order of the factors impacting faculty's attitudes towards e-testing by means of 
their responses 


Factors/Survey sections 

Mean 

Std. Deviation 

Perceived self-efficacy 

3.67 

0.75 

Perceived enjoyment 

3.88 

0.84 

Perceived usefulness 

4.08 

0.88 

Behavioural intention to use e-testing 

3.87 

0.86 

Perceived system satisfaction 

3.64 

0.98 

Perceived system challenges 

3.71 

0.81 


Effect of age. Again, the researcher ran a one-way AN OVA to analyze the differences 
between teachers’ means of their responses to the factors impacting their attitudes towards 
e-testing designated by the six sections of the survey and their associated procedures to see 
if there were any differences between informants on the variable of experience. Results are 
presented in Table 18 below: 
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3.87 


Figure 2. Faculty attitudes towards online testing 


Table 18: ANOVA for the factors affecting faculty's attitudes towards e-testing by age 


Factors impacting 


Sum of 


Mean 



attitudes 

Source of variance 

squares 

df 

square 

F 

Sig. 

Perceived enjoyment 

Between Groups 

40.643 

2 

20.321 

23.096 

.000 


Within Groups 

19.357 

22 

.880 




Total 

60.000 

24 




Perceived usefulness 

Between Groups 

37.153 

2 

18.576 

10.064 

.001 


Within Groups 

40.607 

22 

1.846 




Total 

77.760 

24 




Behavioural intention 

Between Groups 

29.846 

2 

14.923 

8.480 

.002 

to use e-testing 

Within Groups 

38.714 

22 

1.760 




Total 

68.560 

24 




Perceived system 

Between Groups 

32.000 

2 

16.000 

5.029 

.016 

challenges 

Within Groups 

70.000 

22 

3.182 




Total 

102.000 

24 




Perceived system 

Between Groups 

21.042 

2 

10.521 

4.224 

.028 

challenges 

Within Groups 

54.798 

22 

2.491 




Total 

75.840 

24 




Attitudes towards 

online evaluation 

Between Groups 

38.181 

2 

19.091 

5.304 

.013 


As shown in Table i8, the ANOVA tests revealed that the attitudes of teachers towards 
online testing according to the age variable indicate that there is a statistical difference 
between faculty's attitudes towards online testing across teaching/testing experience using 
new technology. 

Discussion 

The expediting diffusion of technology for use in both e-learning and e-testing has made 
available more apt opportunities for students and teachers alike to improve learning 
and achievement outcomes. There is growing enthusiasm to use the new technology as 
a national goal. The Ministry of Higher Education in Saudi Arabia is encouraging, by 
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financial incentives and through providing infrastructure to universities, technology users 
to get acculturated to the new technology atmosphere for both e-leaming and e-testing 
purposes. Therefore, all Saudi universities and teacher training institutions have taken 
positive steps in eguipping pre-service teachers with knowledge and skills in information 
technology. 

Web-based assessment and practice provided students with immediate feedback and 
automated scores that help students to have more control over their work and their effort. 
The students’ survey results related to their perceptions and evaluation of the web-based 
assessment endeavours indicated that both test-takers' and faculty's self-efficacy, enjoyment, 
usefulness, behavioural intentions to use web-based assessment, system satisfaction and 
system challenges in this order appeared to be the most attractive features of the web- 
based learning that formed the attitudes towards e-testing and generated the students' and 
faculty's salient beliefs about the conseguences of continued use and the evaluation. These 
features might also affect the students' success or failure in the acceptance and subseguent 
adoption of e-testing technology, and conseguently in foreign language learning. 

In addition, results from the present study, fortunately commensurate with prior 
research and relevant literature on e-learning and e-testing, have showed that both exter¬ 
nal factors impacting the formation of attitudes towards e-testing, including the social 
environment and learning management systems and the internal factors, including indi¬ 
vidual characteristics of teachers and students are crucial for efficient online assessment 
adoption, implementation, and diffusion. By the same token, a higher level of individual 
computer self-efficacy is positively associated with a higher level of learning performance 
which increases the use of e-learning (Wu, Tennyson, & Hsia, 2010), an observation that 
has also been proven true by the findings of this study. This study has also proven that 
motivation, both internal and instrumental are strongly associated with effective adoption 
of e-testing environments; these findings here are resonant of prior research findings on 
attitudes towards and motivation for web-based assessment. This study also found that per¬ 
ceived self-efficacy, perceived system satisfaction, perceived usefulness and perceived enjoy¬ 
ment have direct positive effects to intention to use an LMS efficiently as they have direct 
positive effects on the intention and motivation to use web-based assessment technology 
for purposes of formative and summative evaluation - findings commensurate with prior 
research (Lee, 2010; Roca and Gagne, 2008). Thus, with the immediate feedback and instant 
scoring, the web-based assessment and practice not only plays the role of measurement or 
evaluation, but it also plays the role of instruction, reflection and reinforcement (Bransford, 
Brown, & Cocking, 1999). Students who take web-based assessment have opportunities to 
show their understanding; additionally, they can learn from their responses or mistakes 
to clarify, review, and reconfirm previous concepts, and finally integrate complex language 
concepts from the multiple practicing. 

Web-based assessment and practice offered students multiple practice opportunities 
that eventually encouraged students to spend more time on tasks and attain higher levels 
of achievement. The feature of the randomized item generation provided by the web-based 
assessment system offered students multiple versions of each homework set. Students could 
practice as many versions or as many times as possible when it is necessary. Along with the 
immediate feedback and automated scoring, the automatic item generation encouraged 
students (a) who highly desire a perfect performance, having an opportunity to reach the 
maximum scores, and (b) who are not confident with their understanding, confirming and 
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However, the challenges facing both students and teachers and shaping their attitudes 
are vast and complicated, and affect them on a personal level. When applying a learn¬ 
ing tool or system for learners, it is necessary to investigate both teachers’ and learners' 
attitudes toward that tool or system. Essentially, understanding their perceptions toward 
learning environments is a crucial issue for enhancing teaching performance and learn¬ 
ing effects. That is why their emotional reactions to the use of technology for teaching 
and learning should be seriously considered. English language teachers must always use 
a variety of tools to produce successful learning experiences and the computer is one of 
them. Technology cannot be sidelined and in this case, the computer and the Internet are 
resources to enhance teaching and testing, and promote the successful performance of 
students. 

The results of this study confirm the assumption that instructors are willing to use 
e-learning environments to conduct assessment activities online. Findings from this study 
also provide evidence that instructors are highly inclined to apply eLearning technology to 
assess achievement and performance in their coursework online, believing that e-testing 
could be also a substantial teaching assistive tool. This is well associated with raising aware¬ 
ness to use technology in higher education learning. 

It was also revealed that long-term retention of knowledge and skills related to technol¬ 
ogy use, content assimilation and preparedness to testing can be maintained in the long¬ 
term memory despite the criticism that drill-and-practice benefits are restricted only to 
short-term memory work (Carpenter & Lehrer, 1999), the reason being that students could 
be interested and more persistent to use online testing drill and practice for preparing for 
important tests. In addition, the study revealed also that when students engaged in the 
web-based assessment and practice, they were willing to spend more time on tasks to gain 
understanding and to strive for better achievement. 

Furthermore, online assessment and e-test-taking practice, using drill-and-practice tuto¬ 
rials were found to enhance test-takers’ confidence in doing their tests online much more 
than on paper-based tests. Additional information from the survey results also demon¬ 
strated that web-based assessment with features of immediate feedback, clear instruction, 
and instant scoring gave students better guidance to direct their learning. 

In addition, web-based language assessment and practice enabled participants, who had 
some level of anxiety in learning a foreign language, to recognize their scores instantly 
through the Grade Centre; thus, they might have more control over their learning, and 
felt less anxious about doing math. It was apparent that the web-based multiple practices 
fostered students' self-assessment and self-regulation in which students could assess what 
they already know and what they need to improve, to make their knowledge base compat¬ 
ible with the learning goals. From this point of view, web-based assessment truly scaffolded 
students' learning processes. 

The results of this study confirm that instructors are willing to use e-testing environ¬ 
ments to convert to web-based assessment activities. The results also provide evidence that 
instructors have positive attitudes to apply e-testing technology in delivering formative 
and summative online, believing that e-testing technology could be an assistive learning 
tool that is complementary to e-learning. This attitude is well suited to raising awareness 
about the use of technology in higher-education learning. The Endings of this study imply 
that enhancing faculty and learners' behaviours and attitudes is reguired for the success 
of future e-testing adoption, implementation and diffusion. 
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Implications 

The findings from this study imply that creating web-based assessment technology aware¬ 
ness, motivation, and changing faculty and learners' behaviours and attitudes is required 
for the success of e-testing adoption, implementation and diffusion in the future. Since 
learners and teachers, too, are habituated to traditional teaching approaches (Miller et al., 
2004) especially in developing countries where ICT is still in its infancy of adoption, some 
Saudi teachers' motivation and attitudes negatively affected by the students' reluctance to 
respond to e-testing activities on Blackboard, perhaps due to the intricacies of the system. 
These findings related to system design are commensurate with research findings from 
Lennon and Maurer (2003) which indicated that system design should be easy to use or 
else it will create confusion among users. 

Furthermore, ongoing training on web-based assessment via LMSs is a necessity for 
new and veteran faculty as well as for students, too. Therefore, it is important to provide 
computer and Internet training to users to become familiar with e-testing technology and 
enhance users’ skills and attitude towards technology. This is consistent with Lee (2008) 
who found that universities' providing computer support and training to learners strongly 
influence a learner’s perceived ease of use and usefulness of the system. More research on 
the influence of web-based assessment and practice on students’ performance should be 
conducted in different levels of pre-tertiary and tertiary education and at different grade 
levels to confirm the actual benefits of the online delivery system. In the future research, 
web-based assessment systems should be implemented for classroom assignments and prac¬ 
tices, and should be assigned to students by their own teachers in addition to the traditional 
assessment over an entire academic year. The measurement of the language achievement 
and attitude should be the results of the long-term study across a diverse population of 
students with varied ages and educational backgrounds. 

Technically, more web-based assessment tasks on various language drills and concepts 
should be designed. The database system should be constructed to manipulate a large- 
scale data collection, and to generate item pools for different language learning levels. An 
implementation of diagnostic systems for different language strands is also suggested to 
enrich the automatic feedback to teachers and educators. 
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